Story Segmentation of Broadcasted Sports Videos with Intermodal Collaboration

This paper investigates the problem of efficiently describing broadcasted sports videos for effective multimedia applications. Considering the sports videos as a sequence of recurrent semantic story units, we propose a method for segmenting the sports videos into the story units and attaching the closed-caption segments, which correspond to the story units, as the detailed descriptions. This proposed method restricts the use of much domain-dependent information and can be used to acquire the semantic content. We first try to segment the closed-caption text into scene units, a set of which comprises a story unit, in a probabilistic framework based on Bayesian Networks. Finding the boundaries of the set of the scene units enables us to generate the story units in the closed-caption text. Template matching in the image stream also segments the video stream into video story units. Finally, temporal association attaches the appropriate closed-caption story unit, which includes the detailed information about semantic content, to each segmented video story unit. We conduct experiments using American football and baseball videos, obtaining successful story segmentation results, a recall rate of 92.5% and a precision rate of 91.5%, and we also discuss the potentiality for utilizing them for a video retrieval system.